Skip to content

Add gen_ai.agent.steps metric - #203

Closed
pvlsirotkin wants to merge 4 commits into
open-telemetry:mainfrom
pvlsirotkin:agent-workflow-steps-metric
Closed

pvlsirotkin wants to merge 4 commits into
open-telemetry:mainfrom
pvlsirotkin:agent-workflow-steps-metric

Conversation

@pvlsirotkin

@pvlsirotkin pvlsirotkin commented May 27, 2026 •

Copy link
Copy Markdown
Contributor

Handoff note: I (@pvlsirotkin) am OOO. @RKest will be driving this PR forward — he has push access to the branch on my fork. Please direct further review feedback to him.

Description

Adds a new GenAI semantic convention metric for the number of steps a GenAI
agent takes during a single invocation, modeled on the recently-merged
gen_ai.workflow.duration metric
(#126):

  • gen_ai.agent.steps (Histogram, {step}): number of discrete steps a
    GenAI agent takes during a single invocation.

The metric counts only steps the agent itself authored (e.g. in ADK, events
whose event.author equals the agent name). Tool results are not counted
(authored by the tool, not the agent). Sub-agent steps are not counted (each
step is recorded exactly once across the full call tree). Failed and partial
steps still count — they consumed the agent's execution.

Step counts MUST NOT be compared across different frameworks; instrumentations
MUST document what counts as a step for their framework.

Motivation

The existing GenAI conventions describe agents and tools but offer no
standardized way to answer "how many steps did this agent take to produce
its response?" Operators today derive that figure ad-hoc from span counts,
which is fragile and depends on sampling configuration.

User journey: capacity planning, regression detection across agent versions,
and diagnosing pathological tool loops (an agent stuck calling tools without
making progress).

Checklist

  • Motivation section filled in above
  • Reference instrumentation and scenarios updated for affected libraries
  • Changelog entry added under changelog.d/ (Towncrier fragment)

See CONTRIBUTING.md.

@linux-foundation-easycla

linux-foundation-easycla Bot commented May 27, 2026 •

Copy link
Copy Markdown

CLA Signed
The committers listed above are authorized under a signed CLA.

  • ✅ login: pvlsirotkin / name: Pavel Sirotkin (aefc0d6)

@pvlsirotkin
pvlsirotkin force-pushed the agent-workflow-steps-metric branch from 7c3176b to 1d31b18 Compare May 27, 2026 10:56
This was referenced May 27, 2026
@pvlsirotkin
pvlsirotkin force-pushed the agent-workflow-steps-metric branch 2 times, most recently from aefc0d6 to d985496 Compare May 27, 2026 15:27
@pvlsirotkin
pvlsirotkin marked this pull request as ready for review May 27, 2026 15:32
@pvlsirotkin
pvlsirotkin requested a review from a team as a code owner May 27, 2026 15:32
Copilot AI review requested due to automatic review settings May 27, 2026 15:32

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Note

Copilot was unable to run its full agentic suite in this review.

Adds a new GenAI workflow metric (gen_ai.workflow.steps) to measure discrete step counts within a workflow execution, with corresponding schema snapshot, registry entry, and documentation updates.

Changes:

  • Introduce gen_ai.workflow.steps histogram metric with agent/workflow dimensions.
  • Update generated schema snapshot to include the new metric.
  • Extend GenAI metrics documentation and changelog to describe the new metric.

Reviewed changes

Copilot reviewed 4 out of 4 changed files in this pull request and generated 3 comments.

File Description
schema-snapshot/registry.yaml Adds the generated registry + refinements entries for gen_ai.workflow.steps.
model/gen-ai/metrics.yaml Defines the new gen_ai.workflow.steps metric and its attributes.
docs/gen-ai/gen-ai-metrics.md Documents gen_ai.workflow.steps and adds it to the TOC.
CHANGELOG.md Notes addition of the new workflow steps metric.

Comment thread schema-snapshot/registry.yaml Outdated
Comment thread schema-snapshot/registry.yaml Outdated
Comment thread model/gen-ai/metrics.yaml Outdated
Comment thread model/gen-ai/metrics.yaml Outdated
@trask trask mentioned this pull request May 30, 2026
3 tasks done
pvlsirotkin added a commit to pvlsirotkin/semantic-conventions-genai that referenced this pull request Jun 2, 2026
* Expand the `gen_ai.workflow.steps` 'step' definition with a list of
  what counts as a step in the most common frameworks (ADK events,
  LangGraph nodes, LangChain AgentAction/AgentFinish, CrewAI agent
  turns) and add a MUST-document requirement for instrumentations.
  Addresses trask's feedback that the original definition was too vague.
* Standardize the conditionally_required phrasing for all three
  attributes on this metric (gen_ai.workflow.name, gen_ai.agent.name,
  gen_ai.agent.version) to 'If available.', matching the adjacent
  gen_ai.workflow.duration entry. Addresses Copilot's within-metric
  inconsistency comment.
pvlsirotkin added a commit to pvlsirotkin/semantic-conventions-genai that referenced this pull request Jun 2, 2026
* Expand the `gen_ai.workflow.steps` 'step' definition with a list of
  what counts as a step in the most common frameworks (ADK events,
  LangGraph nodes, LangChain AgentAction/AgentFinish, CrewAI agent
  turns) and add a MUST-document requirement for instrumentations.
  Addresses trask's feedback that the original definition was too vague.
* Standardize the conditionally_required phrasing for all three
  attributes on this metric (gen_ai.workflow.name, gen_ai.agent.name,
  gen_ai.agent.version) to 'If available.', matching the adjacent
  gen_ai.workflow.duration entry. Addresses Copilot's within-metric
  inconsistency comment.
@pvlsirotkin
pvlsirotkin force-pushed the agent-workflow-steps-metric branch from a5286d5 to 276a553 Compare June 2, 2026 22:17
Comment thread docs/gen-ai/gen-ai-metrics.md Outdated
Comment thread docs/gen-ai/gen-ai-metrics.md Outdated
Comment thread model/gen-ai/metrics.yaml
Comment thread model/gen-ai/metrics.yaml
Comment thread docs/gen-ai/gen-ai-metrics.md Outdated
Comment thread model/gen-ai/metrics.yaml
code_generation:
metric_value_type: int
brief: 'Number of steps in a single GenAI workflow.'
note: |

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do we only count completed steps, or also include partial / failed steps too?

@pvlsirotkin pvlsirotkin Jun 10, 2026 •

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

New answer: tbc
Old answer: Both. The note now says: 'Steps are counted regardless of outcome: a step that errored or partially completed still consumed the agent's execution and SHOULD be counted.'

Comment thread docs/gen-ai/gen-ai-metrics.md Outdated
This metric complements `gen_ai.workflow.duration` by giving operators
visibility into how "chatty" or compute-intensive a workflow is, and
is useful for capacity planning and regression detection across
versions.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

assuming we were able to stamp workflow name on all nested spans and metrics emitted in the context of this workflow, we'd had

  • count of agent invocations per workflow name
  • count of tool calls per workflow name
  • .. (any other metrics, e.g. if we instrumented transfers, we'd had a count of transfers)

Would this metric be still necessary?

In the current shape this metric seems to be an aggregation of multiple independent things which maybe also queried and sumed up individually, but then would provide better observability.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

With the refocus to per-agent scope (see summary) this is no longer a workflow-level aggregation. It's a per-invocation count of an agent's own steps — distinct from summing child signals because (a) not every event is independently traced, and (b) operators get a single histogram they can directly compute percentiles on rather than needing to sum multiple counters and join by attribute.

Comment thread docs/gen-ai/gen-ai-metrics.md Outdated
Comment thread docs/gen-ai/gen-ai-metrics.md Outdated
Comment thread model/gen-ai/metrics.yaml Outdated
Comment thread model/gen-ai/metrics.yaml Outdated
Comment thread model/gen-ai/metrics.yaml Outdated
Comment thread docs/gen-ai/gen-ai-metrics.md Outdated
Adds a new GenAI semantic convention metric that measures the number of
discrete steps observed within a single GenAI workflow:

  * gen_ai.workflow.steps (histogram, {step}): count of observable events
    within one workflow execution. A 'step' is intentionally
    framework-defined - it typically corresponds to one event (a tool
    call, a tool result, a model response in ADK; a node execution in
    LangGraph, etc.).

Dimensioned by gen_ai.workflow.name; when set, gen_ai.agent.name and
gen_ai.agent.version may be used to slice steps per-agent within a
workflow. In that case the count is restricted to events attributed to
that agent.

This metric is colocated with gen_ai.workflow.duration (added in open-telemetry#126)
in the model and docs so the two workflow-level metrics live together.

NOTE: docs/registry/ and schema-snapshot/ regeneration via 'make
generate-all' has NOT been run in this commit (no Docker available in
the authoring environment). Run it locally before pushing for review.
* Expand the `gen_ai.workflow.steps` 'step' definition with a list of
  what counts as a step in the most common frameworks (ADK events,
  LangGraph nodes, LangChain AgentAction/AgentFinish, CrewAI agent
  turns) and add a MUST-document requirement for instrumentations.
  Addresses trask's feedback that the original definition was too vague.
* Standardize the conditionally_required phrasing for all three
  attributes on this metric (gen_ai.workflow.name, gen_ai.agent.name,
  gen_ai.agent.version) to 'If available.', matching the adjacent
  gen_ai.workflow.duration entry. Addresses Copilot's within-metric
  inconsistency comment.
The original RFC and the production ADK implementation always intended
this metric to count steps a *specific agent* takes during a single
invocation - not steps in some larger workflow. A previous revision of
this PR moved the metric into the gen_ai.workflow.* namespace, which
caused most of the confusion in the latest review round (multiple
reviewers asked whether the metric is a per-invocation count or a
cumulative workflow-level aggregation; whether nested sub-agent steps
should be counted; whether workflow.name should be required given the
name implies workflow scope).

This commit refocuses the metric back on its intended scope:

* Rename gen_ai.workflow.steps -> gen_ai.agent.steps. Lives in the
  gen_ai.agent.* namespace alongside gen_ai.agent.invocation.duration
  (open-telemetry#201).
* Drop gen_ai.workflow.name and gen_ai.agent.version from the
  attribute list. The metric is now dimensioned only by
  gen_ai.agent.name (at 'recommended' level, matching open-telemetry#201/open-telemetry#202).
* Add attributes.gen_ai.error ref_group (Mike's feedback).
* Rewrite the metric note to spell out per-agent semantics:
  - Counts only events the agent itself authored (e.g. in ADK,
    events whose event.author equals the agent name).
  - Tool results are NOT counted (authored by tool, not agent).
  - Sub-agent steps are NOT counted (each step recorded once
    across the call tree - addresses aabmass' 'recorded exactly
    once' invariant).
  - Failed and partial steps still count (they consumed execution
    time - addresses Mike's question).
  - Step counts MUST NOT be compared across frameworks (addresses
    Mike's stronger-wording ask).
  - Concrete framework-specific definitions for ADK, LangGraph,
    LangChain agents, CrewAI, with a link to the adk.dev
    event-loop documentation (aabmass' suggestion).
  - Histogram bucket recommendation included in the YAML note
    (Mike's #M3 ask).
* Restructure the docs/gen-ai/gen-ai-metrics.md page: remove
  gen_ai.agent.steps from the 'Generative AI workflow metrics'
  section and create a new 'Generative AI agent metrics' section.
  MD body kept thin; the metric note in YAML carries the full
  semantics and is rendered via the autogen block.
@pvlsirotkin
pvlsirotkin force-pushed the agent-workflow-steps-metric branch from 276a553 to f4f0a4c Compare June 10, 2026 15:18
@pvlsirotkin

Copy link
Copy Markdown
Contributor Author

Big update on this PR after re-reading the original RFC and the production ADK implementation: the metric was always meant to count steps a specific agent takes during a single invocation, not steps in some larger workflow. An earlier revision of this PR pushed it into the gen_ai.workflow.* namespace, which turned out to be the source of most of the confusion in this review round (per-invocation vs cumulative, whether to count sub-agents, whether workflow.name should be required, whether this is just an aggregation of child counts, etc.).

Just pushed a commit that:

The MD page got a new ## Generative AI agent metrics section so the metric no longer lives under "Generative AI workflow metrics". MD body is kept thin; the full semantics live in the YAML note and are rendered into the autogen block.

Replying inline on each open thread.

The bucket-boundaries recommendation was being rendered in two places: in
the MD prose above the autogen block, and inside the YAML `note` (which
gets rendered into the autogen block too). Other duration/usage metrics in
this file keep ExplicitBucketBoundaries only in the MD prose by
convention; aligned with that pattern.

Net effect: the recommendation appears once, in the MD prose, like
`gen_ai.workflow.duration`, `gen_ai.client.token.usage`, and the other
existing metrics.
@trask

trask commented Jun 10, 2026

Copy link
Copy Markdown
Member

Hi! As part of #275, this repository switched to Towncrier changelog fragments to reduce merge conflicts in CHANGELOG.md.

Please move this PR's changelog entry out of CHANGELOG.md and into this Towncrier fragment:

Create changelog.d/203.enhancement.md containing the change log entry, e.g.:

Add `gen_ai.agent.steps` metric to track the number of steps a GenAI agent takes during a single invocation.

After adding the fragment, please remove this PR's direct edit to CHANGELOG.md. Towncrier will add the PR link from the fragment filename when release notes are generated.

Thanks!

the workflow graph.
- **LangChain agents**: each `AgentAction` or `AgentFinish` step the
agent itself emitted.
- **CrewAI**: each agent turn the agent itself took within a task.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I did a bit of a research and I think this generalization would be confusing, it does not make sense to merge these things into one metric unless they are comparable:

  1. For ADK it would count LLM and tool calls, handoffs
  2. Langgraph has a notion of super-step which is "considered a single iteration over the graph nodes" (active ones). So by definition it's more than one node and step used in the metric name would be misleading. Langgraph also has configurable recursion_limit for number of super-steps and remaining_steps in the step that provides number of remaining ones until limit is reached. This is a signal that these things could be meaningful to LangGraph users.
  3. I can't find a term 'turn' in crewai, my AI found iterations in agent executor - https://github.com/crewAIInc/crewAI/blob/b6e5d632c111e098d883274f0a26f02836a2172e/lib/crewai/src/crewai/agents/crew_agent_executor.py#L218 and there is a corresponding config for max_interations, but iteration is a full ReAct loop. It has something called step_callback but does not clearly document when it's called.

I think this metric tries to capture something that's a proxy to number of loops / iterations within one agent, but tries to express it through internal frameowrk implementation details.

Some alternative approaches:

  1. Stamp agent name on the underlying llm and tool calls and derive average number of calls per agent from gen_ai.inference | execute_tool.duration and invoke agent duration. This won't give outliers or percentiles, but arguably many loops / events also leads to long invoke_agent duration.
  2. Measure gen_ai.agent.inference_calls_per_invocation, gen_ai.agent.tool_calls_per_invocations, gen_ai.agent.transfers_per_invocation - these are understandable and generically applicable. The list can probably grow with retrievals and memory calls
  3. Have specialized metrics for frameworks. [gcp.]adk.event_loop.event.count with attributes for author and type or [gcp].adk.event_loop.events_per_invocation would be specific and understandable. Langgraph and crewai would have specialized metrics too

Approaches 1 and 2 can exist together: gen_ai.agent.inference_calls_per_invocation could be a common 'give me a sense of how long this agent invocation was' that shows up on common dashboards and [gcp].adk.event_loop.events_per_invocation can be a detailed breakdown. This would align with t-shaped signals approach.

@pvlsirotkin pvlsirotkin changed the title Add gen_ai.workflow.steps metric Add gen_ai.agent.steps metric Jun 18, 2026
@pvlsirotkin

Copy link
Copy Markdown
Contributor Author

Heads up — I'm OOO from today. @RKest will be taking this PR forward (he has push access to the branch on my fork).

State of the PR:

  • The metric was renamed and refocused: gen_ai.workflow.steps → gen_ai.agent.steps, scoped to counting steps a specific agent itself authored during one invocation (per the underlying RFC and the production ADK implementation). The earlier "workflow" framing caused most of the confusion in prior review rounds.
  • 3 review threads remain open:
    1. Generalization concern — @lmolkova thinks the metric as currently defined isn't comparable across frameworks because what counts as a "step" varies. Needs follow-up.
    2. Workflow-level aggregation alternative — @lmolkova suggested an alternative: if workflow.name could be stamped on all nested signals, this metric might be derivable by query rather than emitted directly. My response defended the per-agent metric as still useful — awaiting her take.
    3. Completed vs partial/failed steps — my reply has a New answer: tbc / Old answer:... artifact from an edit conflict; @RKest should clean this up and confirm the answer.

Background context I'm leaving behind for @RKest:

  • The metric counts events the agent itself authored (e.g. ADK events where event.author == agent_name). Tool results are NOT counted (authored by tool). Sub-agent steps are NOT counted (each step once across the call tree).
  • Step counts MUST NOT be compared across frameworks; instrumentations MUST document what they count.
  • This was the source of a lot of cross-talk between @trask, @MikeGoldsmith, @lmolkova, @aabmass — recommend reading the top-level summary comment first for context.

Thanks everyone for the feedback so far.

@Necmttn

This comment has been minimized.

RKest added a commit to RKest/semantic-conventions-genai that referenced this pull request Jun 23, 2026
Following review feedback on open-telemetry#203, the single gen_ai.agent.steps metric
conflated things that are not comparable across frameworks (LLM calls,
tool calls, handoffs, framework-specific 'super-steps'/'iterations'),
which made the 'step' unit misleading and framework-dependent.

This replaces it with two understandable, generically-applicable
per-invocation metrics:

* gen_ai.agent.inference_calls_per_invocation (histogram,
  {inference_call}): number of inference (model) calls an agent makes
  during a single invocation.
* gen_ai.agent.tool_calls_per_invocation (histogram, {tool_call}):
  number of tool calls an agent makes during a single invocation.

Both are dimensioned by gen_ai.agent.name (recommended) and live in the
gen_ai.agent.* namespace alongside gen_ai.invoke_agent.duration. Counts
are scoped to a single invocation and counted exactly once across the
call tree (sub-agent/transferred calls belong to those agents'
invocations). Failed and partial calls still count.

gen_ai.agent.transfers_per_invocation (also suggested in the review) is
intentionally omitted: a transfer/handoff count is not a good measure of
an agent's workflow complexity. The list can still grow later (e.g.
retrievals, memory calls).
RKest added a commit to RKest/semantic-conventions-genai that referenced this pull request Jun 23, 2026
Following review feedback on open-telemetry#203, the single gen_ai.agent.steps metric
conflated things that are not comparable across frameworks (LLM calls,
tool calls, handoffs, framework-specific 'super-steps'/'iterations'),
which made the 'step' unit misleading and framework-dependent.

This replaces it with two understandable, generically-applicable
per-invocation metrics:

* gen_ai.agent.inference_calls_per_invocation (histogram,
  {inference_call}): number of inference (model) calls an agent makes
  during a single invocation.
* gen_ai.agent.tool_calls_per_invocation (histogram, {tool_call}):
  number of tool calls an agent makes during a single invocation.

Both are dimensioned by gen_ai.agent.name (recommended) and live in the
gen_ai.agent.* namespace alongside gen_ai.invoke_agent.duration. Counts
are scoped to a single invocation and counted exactly once across the
call tree (sub-agent/transferred calls belong to those agents'
invocations). Failed and partial calls still count.

gen_ai.agent.transfers_per_invocation (also suggested in the review) is
intentionally omitted: a transfer/handoff count is not a good measure of
an agent's workflow complexity. The list can still grow later (e.g.
retrievals, memory calls).
RKest added a commit to RKest/semantic-conventions-genai that referenced this pull request Jun 23, 2026
Following review feedback on open-telemetry#203, the single gen_ai.agent.steps metric
conflated things that are not comparable across frameworks (LLM calls,
tool calls, handoffs, framework-specific 'super-steps'/'iterations'),
which made the 'step' unit misleading and framework-dependent.

This replaces it with two understandable, generically-applicable
per-invocation metrics:

* gen_ai.agent.inference_calls_per_invocation (histogram,
  {inference_call}): number of inference (model) calls an agent makes
  during a single invocation.
* gen_ai.agent.tool_calls_per_invocation (histogram, {tool_call}):
  number of tool calls an agent makes during a single invocation.

Both are dimensioned by gen_ai.agent.name (recommended) and live in the
gen_ai.agent.* namespace alongside gen_ai.invoke_agent.duration. Counts
are scoped to a single invocation and counted exactly once across the
call tree (sub-agent/transferred calls belong to those agents'
invocations). Failed and partial calls still count.

gen_ai.agent.transfers_per_invocation (also suggested in the review) is
intentionally omitted: a transfer/handoff count is not a good measure of
an agent's workflow complexity. The list can still grow later (e.g.
retrievals, memory calls).
RKest added a commit to RKest/semantic-conventions-genai that referenced this pull request Jun 23, 2026
Following review feedback on open-telemetry#203, the single gen_ai.agent.steps metric
conflated things that are not comparable across frameworks (LLM calls,
tool calls, handoffs, framework-specific 'super-steps'/'iterations'),
which made the 'step' unit misleading and framework-dependent.

This replaces it with two understandable, generically-applicable
per-invocation metrics:

* gen_ai.agent.inference_calls_per_invocation (histogram,
  {inference_call}): number of inference (model) calls an agent makes
  during a single invocation.
* gen_ai.agent.tool_calls_per_invocation (histogram, {tool_call}):
  number of tool calls an agent makes during a single invocation.

Both are dimensioned by gen_ai.agent.name (recommended) and live in the
gen_ai.agent.* namespace alongside gen_ai.invoke_agent.duration. Counts
are scoped to a single invocation and counted exactly once across the
call tree (sub-agent/transferred calls belong to those agents'
invocations). Failed and partial calls still count.

gen_ai.agent.transfers_per_invocation (also suggested in the review) is
intentionally omitted: a transfer/handoff count is not a good measure of
an agent's workflow complexity. The list can still grow later (e.g.
retrievals, memory calls).
@RKest

RKest commented Jun 23, 2026

Copy link
Copy Markdown
Contributor

Some alternative approaches:
...

  1. Measure gen_ai.agent.inference_calls_per_invocation, gen_ai.agent.tool_calls_per_invocations, gen_ai.agent.transfers_per_invocation - these are understandable and generically applicable. The list can probably grow with retrievals and memory calls
    ...

I went ahead and implemented this alternative in #336

@lmolkova

Copy link
Copy Markdown
Member

I'm going to close this PR since it's continued in #336, feel free to reopen if necessary

@lmolkova lmolkova closed this Jun 24, 2026
RKest added a commit to RKest/semantic-conventions-genai that referenced this pull request Jun 24, 2026
Following review feedback on open-telemetry#203, the single gen_ai.agent.steps metric
conflated things that are not comparable across frameworks (LLM calls,
tool calls, handoffs, framework-specific 'super-steps'/'iterations'),
which made the 'step' unit misleading and framework-dependent.

This replaces it with two understandable, generically-applicable
per-invocation metrics:

* gen_ai.invoke_agent.inference_calls (histogram, {inference_call}):
  number of inference (model) calls an agent makes during a single
  invocation.
* gen_ai.invoke_agent.tool_calls (histogram, {tool_call}): number of
  tool calls an agent makes during a single invocation.

Both are dimensioned by gen_ai.agent.name (recommended) and live in the
gen_ai.invoke_agent.* namespace alongside gen_ai.invoke_agent.duration.
Counts are scoped to a single invocation and counted exactly once across
the call tree (sub-agent/transferred calls belong to those agents'
invocations). Failed and partial calls still count. Both SHOULD be
emitted together with the gen_ai.invoke_agent.internal span.

A transfers/handoffs count (also suggested in the review) is
intentionally omitted: it is not a good measure of an agent's workflow
complexity. The list can still grow later (e.g. retrievals, memory
calls).
RKest added a commit to RKest/semantic-conventions-genai that referenced this pull request Jun 24, 2026
Following review feedback on open-telemetry#203, the single gen_ai.agent.steps metric
conflated things that are not comparable across frameworks (LLM calls,
tool calls, handoffs, framework-specific 'super-steps'/'iterations'),
which made the 'step' unit misleading and framework-dependent.

This replaces it with two understandable, generically-applicable
per-invocation metrics:

* gen_ai.invoke_agent.inference_calls (histogram, {inference_call}):
  number of inference (model) calls an agent makes during a single
  invocation.
* gen_ai.invoke_agent.tool_calls (histogram, {tool_call}): number of
  tool calls an agent makes during a single invocation.

Both are dimensioned by gen_ai.agent.name (recommended) and live in the
gen_ai.invoke_agent.* namespace alongside gen_ai.invoke_agent.duration.
Counts are scoped to a single invocation and counted exactly once across
the call tree (sub-agent/transferred calls belong to those agents'
invocations). Failed and partial calls still count. Both SHOULD be
emitted together with the gen_ai.invoke_agent.internal span.

A transfers/handoffs count (also suggested in the review) is
intentionally omitted: it is not a good measure of an agent's workflow
complexity. The list can still grow later (e.g. retrievals, memory
calls).
RKest added a commit to RKest/semantic-conventions-genai that referenced this pull request Jun 24, 2026
Following review feedback on open-telemetry#203, the single gen_ai.agent.steps metric
conflated things that are not comparable across frameworks (LLM calls,
tool calls, handoffs, framework-specific 'super-steps'/'iterations'),
which made the 'step' unit misleading and framework-dependent.

This replaces it with two understandable, generically-applicable
per-invocation metrics:

* gen_ai.invoke_agent.inference_calls (histogram, {inference_call}):
  number of inference (model) calls an agent makes during a single
  invocation.
* gen_ai.invoke_agent.tool_calls (histogram, {tool_call}): number of
  tool calls an agent makes during a single invocation.

Both are dimensioned by gen_ai.agent.name (recommended) and live in the
gen_ai.invoke_agent.* namespace alongside gen_ai.invoke_agent.duration.
Counts are scoped to a single invocation and counted exactly once across
the call tree (sub-agent/transferred calls belong to those agents'
invocations). Failed and partial calls still count. Both SHOULD be
emitted together with the gen_ai.invoke_agent.internal span.

A transfers/handoffs count (also suggested in the review) is
intentionally omitted: it is not a good measure of an agent's workflow
complexity. The list can still grow later (e.g. retrievals, memory
calls).
RKest added a commit to RKest/semantic-conventions-genai that referenced this pull request Jun 25, 2026
Following review feedback on open-telemetry#203, the single gen_ai.agent.steps metric
conflated things that are not comparable across frameworks (LLM calls,
tool calls, handoffs, framework-specific 'super-steps'/'iterations'),
which made the 'step' unit misleading and framework-dependent.

This replaces it with two understandable, generically-applicable
per-invocation metrics:

* gen_ai.invoke_agent.inference_calls (histogram, {inference_call}):
  number of inference (model) calls an agent makes during a single
  invocation.
* gen_ai.invoke_agent.tool_calls (histogram, {tool_call}): number of
  tool calls an agent makes during a single invocation.

Both are dimensioned by gen_ai.agent.name (recommended) and live in the
gen_ai.invoke_agent.* namespace alongside gen_ai.invoke_agent.duration.
Counts are scoped to a single invocation and counted exactly once across
the call tree (sub-agent/transferred calls belong to those agents'
invocations). Failed and partial calls still count. Both SHOULD be
emitted together with the gen_ai.invoke_agent.internal span.

A transfers/handoffs count (also suggested in the review) is
intentionally omitted: it is not a good measure of an agent's workflow
complexity. The list can still grow later (e.g. retrievals, memory
calls).
copybara-service Bot pushed a commit to google/adk-python that referenced this pull request Jun 26, 2026
Specify explicit bucket boundaries for GenAI metrics histograms to align with OTel semconv updates:
- open-telemetry/semantic-conventions-genai#201
- open-telemetry/semantic-conventions-genai#202
- open-telemetry/semantic-conventions-genai#203

PiperOrigin-RevId: 938553240
copybara-service Bot pushed a commit to google/adk-python that referenced this pull request Jul 6, 2026
Semconv PR for this metric in:
open-telemetry/semantic-conventions-genai#203
Was replaced by `gen_ai.invoke_agent.{inference,tool}_calls` metrics in:
open-telemetry/semantic-conventions-genai#336

We're hence removing the non-standard and less defined
`gen_ai.agent.workflow.steps` metric.

Co-authored-by: Max Ind <maxind@google.com>
PiperOrigin-RevId: 943154162
RKest added a commit to RKest/semantic-conventions-genai that referenced this pull request Jul 6, 2026
Following review feedback on open-telemetry#203, the single gen_ai.agent.steps metric
conflated things that are not comparable across frameworks (LLM calls,
tool calls, handoffs, framework-specific 'super-steps'/'iterations'),
which made the 'step' unit misleading and framework-dependent.

This replaces it with two understandable, generically-applicable
per-invocation metrics:

* gen_ai.invoke_agent.inference_calls (histogram, {inference_call}):
  number of inference (model) calls an agent makes during a single
  invocation.
* gen_ai.invoke_agent.tool_calls (histogram, {tool_call}): number of
  tool calls an agent makes during a single invocation.

Both are dimensioned by gen_ai.agent.name (recommended) and live in the
gen_ai.invoke_agent.* namespace alongside gen_ai.invoke_agent.duration.
Counts are scoped to a single invocation and counted exactly once across
the call tree (sub-agent/transferred calls belong to those agents'
invocations). Failed and partial calls still count. Both SHOULD be
emitted together with the gen_ai.invoke_agent.internal span.

A transfers/handoffs count (also suggested in the review) is
intentionally omitted: it is not a good measure of an agent's workflow
complexity. The list can still grow later (e.g. retrievals, memory
calls).
lmolkova pushed a commit to lmolkova/semantic-conventions-genai that referenced this pull request Jul 17, 2026
…en-telemetry#336)

Following review feedback on open-telemetry#203, the single gen_ai.agent.steps metric
conflated things that are not comparable across frameworks (LLM calls,
tool calls, handoffs, framework-specific 'super-steps'/'iterations'),
which made the 'step' unit misleading and framework-dependent.

This replaces it with two understandable, generically-applicable
per-invocation metrics:

* gen_ai.invoke_agent.inference_calls (histogram, {inference_call}):
  number of inference (model) calls an agent makes during a single
  invocation.
* gen_ai.invoke_agent.tool_calls (histogram, {tool_call}): number of
  tool calls an agent makes during a single invocation.

Both are dimensioned by gen_ai.agent.name (recommended) and live in the
gen_ai.invoke_agent.* namespace alongside gen_ai.invoke_agent.duration.
Counts are scoped to a single invocation and counted exactly once across
the call tree (sub-agent/transferred calls belong to those agents'
invocations). Failed and partial calls still count. Both SHOULD be
emitted together with the gen_ai.invoke_agent.internal span.

A transfers/handoffs count (also suggested in the review) is
intentionally omitted: it is not a good measure of an agent's workflow
complexity. The list can still grow later (e.g. retrievals, memory
calls).
Mohnish-Srivats pushed a commit to Mohnish-Srivats/semantic-conventions-genai that referenced this pull request Jul 30, 2026
…en-telemetry#336)

Following review feedback on open-telemetry#203, the single gen_ai.agent.steps metric
conflated things that are not comparable across frameworks (LLM calls,
tool calls, handoffs, framework-specific 'super-steps'/'iterations'),
which made the 'step' unit misleading and framework-dependent.

This replaces it with two understandable, generically-applicable
per-invocation metrics:

* gen_ai.invoke_agent.inference_calls (histogram, {inference_call}):
  number of inference (model) calls an agent makes during a single
  invocation.
* gen_ai.invoke_agent.tool_calls (histogram, {tool_call}): number of
  tool calls an agent makes during a single invocation.

Both are dimensioned by gen_ai.agent.name (recommended) and live in the
gen_ai.invoke_agent.* namespace alongside gen_ai.invoke_agent.duration.
Counts are scoped to a single invocation and counted exactly once across
the call tree (sub-agent/transferred calls belong to those agents'
invocations). Failed and partial calls still count. Both SHOULD be
emitted together with the gen_ai.invoke_agent.internal span.

A transfers/handoffs count (also suggested in the review) is
intentionally omitted: it is not a good measure of an agent's workflow
complexity. The list can still grow later (e.g. retrievals, memory
calls).
FrigaZzz pushed a commit to FrigaZzz/adk-python that referenced this pull request Aug 11, 2026
Specify explicit bucket boundaries for GenAI metrics histograms to align with OTel semconv updates:
- open-telemetry/semantic-conventions-genai#201
- open-telemetry/semantic-conventions-genai#202
- open-telemetry/semantic-conventions-genai#203

PiperOrigin-RevId: 938553240
FrigaZzz pushed a commit to FrigaZzz/adk-python that referenced this pull request Aug 11, 2026
Semconv PR for this metric in:
open-telemetry/semantic-conventions-genai#203
Was replaced by `gen_ai.invoke_agent.{inference,tool}_calls` metrics in:
open-telemetry/semantic-conventions-genai#336

We're hence removing the non-standard and less defined
`gen_ai.agent.workflow.steps` metric.

Co-authored-by: Max Ind <maxind@google.com>
PiperOrigin-RevId: 943154162
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Development

Successfully merging this pull request may close these issues.

8 participants